This work proposes a universal and adaptive second-order method for minimizing second-order smooth, convex functions. Our algorithm achieves $O(\sigma / \sqrt{T})$ convergence when the oracle feedback is stochastic with variance $\sigma^2$, and improves its convergence to $O( 1 / T^3)$ with deterministic oracles, where $T$ is the number of iterations. Our method also interpolates these rates without knowing the nature of the oracle apriori, which is enabled by a parameter-free adaptive step-size that is oblivious to the knowledge of smoothness modulus, variance bounds and the diameter of the constrained set. To our knowledge, this is the first universal algorithm with such global guarantees within the second-order optimization literature.

当学习者与其他优化代理进行连续游戏时，我们研究了遗憾最小化的问题：在这种情况下，如果所有玩家都遵循一种无重组算法，则相对于完全对手环境，可能会达到较低的遗憾。我们在变异稳定的游戏（包括所有凸孔和单调游戏的连续游戏）的背景下研究了这个问题，当玩家只能访问其个人回报梯度时。如果噪音是加性的，那么游戏理论和纯粹的对抗性设置也会获得类似的遗憾保证。但是，如果噪声是乘法的，我们表明学习者实际上可以持续遗憾。我们通过学习速率分离的乐观梯度方案实现了更快的速度 - 也就是说，该方法的外推和更新步骤被调整为不同的时间表，具体取决于噪声配置文件。随后，为了消除对精致的超参数调整的需求，我们提出了一种完全自适应的方法，可以在最坏的和最佳案例的遗憾保证之间平稳地插入。

图形卷积网络（GCN）是最受欢迎的体系结构之一，用于解决分类问题，并附有图形信息。我们对图形卷积在多层网络中的影响进行了严格的理论理解。我们通过与随机块模型结合的非线性分离高斯混合模型的节点分类问题研究这些效果。首先，我们表明，单个图卷积扩展了多层网络可以至少$ 1/\ sqrt [4] {\ Mathbb {e} {\ rm veg对数据进行分类的均值之间的距离。 }} $，其中$ \ mathbb {e} {\ rm deg} $表示节点的预期度。其次，我们表明，随着图的密度稍强，两个图卷积将此因素提高到至少$ 1/\ sqrt [4] {n} $，其中$ n $是图中的节点的数量。最后，我们对网络层中不同组合的图形卷积的性能提供了理论和经验见解，得出的结论是，对于所有位置的所有组合，性能都是相互相似的。我们对合成数据和现实世界数据进行了广泛的实验，以说明我们的结果。

Graph-based learning is a rapidly growing sub-field of machine learning with applications in social networks, citation networks, and bioinformatics. One of the most popular models is graph attention networks. They were introduced to allow a node to aggregate information from features of neighbor nodes in a non-uniform way, in contrast to simple graph convolution which does not distinguish the neighbors of a node. In this paper, we study theoretically this expected behaviour of graph attention networks. We prove multiple results on the performance of graph attention mechanism for the problem of node classification for a contextual stochastic block model. Here the node features are obtained from a mixture of Gaussians and the edges from a stochastic block model. We show that in an "easy" regime, where the distance between the means of the Gaussians is large enough, graph attention is able to distinguish inter-class from intra-class edges, and thus it maintains the weights of important edges and significantly reduces the weights of unimportant edges. Consequently, we show that this implies perfect node classification. In the "hard" regime, we show that every attention mechanism fails to distinguish intra-class from inter-class edges. We evaluate our theoretical results on synthetic and real-world data.